In automated software testing, a test program can read a test configuration file that describes test cases to be performed. Each test case defines the setting of the hardware and software in a computer system in which a test is to be performed. For example, a test case can define drive format, Advanced Configuration and Power Interface (ACPI) support, Symmetric Multiprocessing (SMP) configuration, operating system (OS) type and configuration, the number of central processing units (CPUs) to be enabled, and the like. The test can be used to verify the function of a computer system, or a guest that runs in a virtual machine system.
Generally, the software and hardware settings defined in the test cases can be grouped into a number of test categories, and each test category provides a number of options. One example of a test category (category I) is the number of CPUs enabled in a test. The options provided by this test category can include one CPU, two CPUs, three CPUs and four CPUs. Another example of a test category (category II) can be the OS type, which provides the options of Windows®, Linux®, Solaris®, Mac® OS. A test case can be generated by taking an element of the Cartesian product of category I and category II; that is, taking one option from each of category I and category II. The number of elements of the Cartesian product generated from category I and category II is 16, which means that there are 16 different combinations of the options in category I and category II. Each of the combinations represents a specified number of CPUs with a particular type of OS. Therefore, 16 test cases can be generated from these two categories.
To fully test a computer system, a large number of categories are often considered. Therefore, the number of test cases can grow exponentially with the number of categories. However, in some scenarios, not all of these test cases are valid. For example, if a certain type of OS (e.g., Test_OS) can only run on one CPU, then it is not necessary to generate test cases that combines Test_OS with two CPUs, three CPUs, or four CPUs. A conventional automatic test generator can first generate all of the combinations from the categories, and then remove those combinations that are not valid. However, generating all of the combinations requires a significant amount of memory and runtime.